Multi-Task Reinforcement Learning Using Hierarchical Bayesian Models
نویسنده
چکیده
For this project, the objective was to build a working implementation of a multi-task reinforcement learning (MTRL) agent using a hierarchical Bayesian model (HBM) framework described in the paper “Multitask reinforcement learning: A hierarchical Bayesian approach” (Wilson, et al. 2007). This agent was then to play a modified version of the game of Pacman. In this version of the classic arcade game, a series of episodes are played in sequence and the properties of the map in each episode can differ significantly. The end goal was to show that the new agent performs better than a standard Q-learning agent in this version of the game which presents different types of maps which may have different optimal policies. These goals were partially met, but the more advanced algorithms which allow the HBM agent to be extremely adaptable will require more time to implement correctly. However, the initial results clearly show the benefits of using an HBM agent in the MTRL setting.
منابع مشابه
Bayesian Hierarchical Reinforcement Learning
We describe an approach to incorporating Bayesian priors in the MAXQ framework for hierarchical reinforcement learning (HRL). We define priors on the primitive environment model and on task pseudo-rewards. Since models for composite tasks can be complex, we use a mixed model-based/model-free learning approach to find an optimal hierarchical policy. We show empirically that (i) our approach resu...
متن کاملBayesian Multi-Task Reinforcement Learning
We consider the problem of multi-task reinforcement learning where the learner is provided with a set of tasks, for which only a small number of samples can be generated for any given policy. As the number of samples may not be enough to learn an accurate evaluation of the policy, it would be necessary to identify classes of tasks with similar structure and to learn them jointly. We consider th...
متن کاملBayesian Multi-Task Reinforcement Learning
We consider the problem of multi-task reinforcement learning where the learner is provided with a set of tasks, for which only a small number of samples can be generated for any given policy. As the number of samples may not be enough to learn an accurate evaluation of the policy, it would be necessary to identify classes of tasks with similar structure and to learn them jointly. We consider th...
متن کاملMonte carlo bayesian hierarchical reinforcement learning
In this paper, we propose to use hierarchical action decomposition to make Bayesian model-based reinforcement learning more efficient and feasible in practice. We formulate Bayesian hierarchical reinforcement learning as a partially observable semi-Markov decision process (POSMDP). The main POSMDP task is partitioned into a hierarchy of POSMDP subtasks; lower-level subtasks get solved first, th...
متن کاملCombining Hierarchical Reinforcement Learning and Bayesian Networks for Natural Language Generation in Situated Dialogue
Language generators in situated domains face a number of content selection, utterance planning and surface realisation decisions, which can be strictly interdependent. We therefore propose to optimise these processes in a joint fashion using Hierarchical Reinforcement Learning. To this end, we induce a reward function for content selection and utterance planning from data using the PARADISE fra...
متن کامل